Compressed Sensing over ℓp-balls: Minimax mean square error

We consider the compressed sensing problem, where the object x0 ∈ R is to be recovered from incomplete measurements y = Ax0 + z; here the sensing matrix A is an n × N random matrix with iid Gaussian entries and n < N . A popular method of sparsity-promoting reconstruction is `-penalized least-squares reconstruction (aka LASSO, Basis Pursuit). It is currently popular to consider the strict sparsity model, where the object x0 is nonzero in only a small fraction of entries. In this paper, we instead consider the much more broadly applicable `p-sparsity model, where x0 is sparse in the sense of having `p norm bounded by ξ ·N for some fixed 0 < p ≤ 1 and ξ > 0. We study an asymptotic regime in which n and N both tend to infinity with limiting ratio n/N = δ ∈ (0, 1), both in the noisy (z 6= 0) and noiseless (z = 0) cases. Under weak assumptions on x0, we are able to precisely evaluate the worst-case asymptotic minimax mean-squared reconstruction error (AMSE) for ` penalized least-squares: min over penalization parameters, max over `p-sparse objects x0. We exhibit the asymptotically least-favorable object (hardest sparse signal to recover) and the maximin penalization. In the case where n/N tends to zero slowly – i.e. extreme undersampling – our formulas (normalized for comparison) say that the minimax AMSE of `1 penalized least-squares is asymptotic to ξ · ( 2 log(N/n) n ) 2/p−1 · (1 + o(1)). Thus we have not only the rate but also the constant factor on the AMSE; and the maximin penalty factor needed to attain this performance is also precisely specified. Other similarly precise calculations are showcased. Our explicit formulas unexpectedly involve quantities appearing classically in statistical decision theory. Occurring in the present setting, they reflect a deeper connection between penalized ` minimization and scalar soft thresholding. This connection, which follows from earlier work of the authors and collaborators on the AMP iterative thresholding algorithm, is carefully explained. Our approach also gives precise results under weak-`p ball coefficient constraints, as we show here.